Resources for Visual Backpropagation (VBp)

About Visual Backpropagation
Visual Backpropagation is an open source(Apache 2.0 license), functional programming specification of backpropagation that yields a visual and transparent implementation within spreadsheets. This backpropagation implementation exploits array worksheet formulas, manual calculation, and has a sequential order of computation similar to the processing of a systolic array. The implementation uses no hidden macros nor user-defined functions; there are no loops, assignment statements, or links to any procedural programs written in conventional languages. (Other Inductive products)

Recommended citation
Freedman, Roy S. Visual Backpropagation . arXiv:1906.04011 [cs.LG] (2019).
URL: https://arxiv.org/abs/1906.04011

License

Excel VBp Workbooks

• Simple nonlinear regression: learning y=sin(pi*x) via 1-3-1 tanh network (2020).

• The XOR Problem (simple examples of a function not linearly separable): See Freedman, Roy S. Copula Representations and Error Surface Projections for the Exclusive Or Problem. arXiv:1907.04483 [cs.LG] (2019).
  1. 2-2-2-2 with tanh
     Figure 3.1 and Figure 3.2 in Visual Backpropagation .
  2. 2-9-1 vs 2-4-4-1
     Are two hidden layers better than a one big hidden layer?
     Both have the same activation functions -- tanh -- and same number (37) of weight components -- (4x3 + 4x5 + 1x5) vs. (9x3 + 1x10).
     Vbp training sheets and results for 2-4-4-1 (two hidden layers) after 250 epochs; 2-9-1 (one hidden layer) after 1250 epochs.

• The AUTO MPG Problem: Discussed in Section 4 of Visual Backpropagation .
  Citation: Dua, D. and Graff, C. (2019). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.
  1. Data selection, Scaling, Normalization, and Comparisons
  2. 9-50-30-1: RELU-RELU-Id: 120 epochs

(c) 2019 Inductive Solutions, Inc. Except as otherwise noted, the content linked on this page is licensed under the Creative Commons Attribution 4.0 License, and spreadsheet code samples are licensed under the Apache 2.0 License.